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Statewide Proficiency Testing: 
Establishing Standards or Barriers? 



Introduction 

In 1987, the Ohio Legislature passed Substitute House 
Bill 231 which established a statewide high school proficiency 
testing program for Ohio's high school students. The program 
specified that, commencing in 1990, ninth grade students would be 
tested in the areas of reading, mathematics, citizenship, and 
writing. Each student's performance on this test would be one of 
the criteria that determine the certificate or diploma awarded to 
the student at the completion of grade twelve. During the period 
of time between 1987 and fall, 1990 test development proceeded in 
the four subject areas. Learning outcomes were adopted, item 
development and finalization commenced, and the standard setting 
and scoring procedures were completed. The ninth grade class of 
1990-91 (which would graduate in spring, 1994) was tested in 
November, 1990 and again in March, 1991 (if necessary) as the 
first participants in the proficiency program. All public school 
students in the ninth grade in the state were designated to 
participate with exemptions and waivers available under certain 
circumstances. (Private school pupils could participate 
optionally.) During the spring session (1991) tenth through 
twelfth grade students also participated. 

During the preparation period, numerous discussions and 
debates arose concerning the intended use of the test results. 
Students would be required to pass all four parts of the test in 
order to obtain a diploma at graduation. Two opportunities per 
year wojld be given to each student to retake any part of the 
test not passed to date. However, the perceived difficulty level 
of the tests and the pass/fail standard setting process raised 
many questions about the intended practice of withholding 
diplomas of failing students. On one hand, the testing program 
could be regarded as an attempt to maintain minimum standards and 
'guarantee' that certain skills be evident among Ohio's 
graduates, while on the other hand the awarding of a certificate 
of attendance (for those not passing) versus a diploma was raised 
as a discriminatory move against educationally deprived students 
in the state's urban (inner city) and rural districts. 



Questions 

This study was proposed in order to address the following 
questions concerning the above mentioned issues related to the 
testing program. 
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1. Are the tests a valid measure of ninth grade skills 
in the specific subject areas? 

2. Do the results of the first year of testing indicate 
any disparate results between districts of varying 
socioeconomic constituencies? 

Purpose 

The purpose of this study was to explore correlations of 
the performance of students on the state ninth grade proficiency 
test with other measures of achievement in the areas of reading 
and mathematics. Secondly, an inquiry into the performance of 
students from higher versus lower poverty level school districts 
was conducted to address the socioeconomic status question. 



Background 

The Cleveland City School District numbers approximately 
70,000 students in grades kindergarten through twelve. The 
district student population is 70 percent black, 25 percent 
white, and five percent Hispanic, Asian, and others. Seventy 
percent of the enrollment qualifies for the free or reduced lunch 
program indicating the economic status of th<* student population. 

Approximately 6,000 students were enrolled in the ninth 
grade in the Cleveland City School District for the 1990-91 
school year. About 4,200 of these students were tested during 
the first session in November (retesting added about 400 student 
records in March, 1991). Likewise, students in the district were 
tested in the areas of reading and mathematics during the current 
and preceeding academic years. Both norm referenced and 
criterion referenced scores in reading and mathematics were 
obtained as local achievement measures through the district's 
annual testing program while the students were either ninth or 
eighth graders. 

Methodology 

Both questions were addressed utilizing a correlation 
design. Initially, the published learning outcomes (upon which 
the Ohio Ninth Grade Proficiency test is based) were reviewed and 
compared to the published pupil performance objectives and skill 
categories covered by the criterion referenced and norm 
referenced tests utilized in the district. The degree of 
comparability would support the validity of the correlations 
obtained from analyzing the test results. State proficiency test 
results (reported as pass-fail) were correlated with norm 
referenced indices (stanine scores) and criterion referenced 
indices (skill mastery rates) for students with complete results 
on file. 
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Additionally, to address the second question, selected 
district performances reported by the state were correlated with 
the respective districts' published poverty rate (compiled by the 
Council of Economic Opportunity of Greater Cleveland). 

Results 

Learning Outcomes, Pupil performance Objectives and Category 
Objectives 

The Cleveland City School District conducts an annual 
testing program which includes the assessment of pupil 
achievement in the area of reading for pupils in all grades. The 
Cleveland Reading Test was developed by CTB/McGraw-Hill in 
conjunction with the school district in order to utilize one 
testing instrument which simultaneously addressed the district's 
norm referenced test (NRT) requirements and the district's own 
competency objectives (PPO's) via a criterion referenced test 
(CRT). In order to review the objectives addressed by the three 
instruments mentioned above, a direct comparison of the three 
published sets of objectives/outcomes was made. Specifically, 
the outcomes addressed by the ninth grade proficiency test were 
cross referenced with the category objectives addressed by the 
California Achievement Test (CAT Form E/F, Level 18) and the 
eighth and ninth grade PPO's. Twenty four reading outcomes were 
specified by the state for proficiency testing purposes. The 
district specifies 15 eighth grade and 16 ninth grade reading 
objectives for competency assessment while the CAT tests classify 
items by 11 major and 18 subskill categories for eighth grade 
achievement testing. The comparison found that 20 of the state's 
24 outcomes were matched by either the CAT objectives or the 
PPO's tested for competency purposes. The cross-referenced table 
follows (Table 1). 
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TABLE 1 



Reading objectives Cross-referenced Chart 



Ohio Proficiency 
Test Skill 
Objective 



California 
Achievement Test 
(Level , objective) 



Cleveland 
Competency Test 
(Grade,objective) 



Fictions 

Unfamialiar word meaning 

Multi-meaning words 

Details 

Sequences 

Main ideas 

Most probable outcome 
Cause-effect 
Statements of fact/ 

opinion 
Predeictions 
Comprehension of main 

idea & details 

Non-Fiction: 

Details 

Main idea 

Fact-inference 

Fact or opinion 

Predictions 

Details wrt main idea 

Author's purpose 

Summary for audience 

Author's attitude 

Di rections 

Reference materials 

Skills — charts, maps, etc . 

Vocabulary on applications 

Propaganda 



(18,28) 
(18,24) 
(18,32) 
(18,32) 
(18,36) 
(18,37) 
(18,37) 

( n/a ) 
(18,37) 

(18,36) 



(18,32) 
(18,36) 
(18,37) 
(18,38) 
(18,37) 
(18,32) 
(18,36) 
(18,39) 
(18,36) 



n/a 
n/a 
n/a 
n/a 
n/a 



(8,3) 

(8,3) 

(8,12) 

(8,13) 

<8,7)(9,8) 

(8,15) 

(8,5)(9,11) 

(8,4) 
(8,15) 

(9,8) 



(9,8) 

(8,7)(9,8) 

(8,4X9,11) 

(9,15) 

(8,15) , 

(9,14) 

(n/a) 

(9,9) 

(n/a) 

(n/a) 

(n/a) 

(n/a) 

(n/a) 

(8,14) 



Similarly, the state has specified 16 mathematics 
learning outcomes for proficiency testing. In the case of 
mathematics, all 16 outcomes are matched by either CAT objectives 
or Competency objectives specified for eighth grade testing. 
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TABLE 2 

Mathematics Objectives Cross-referenced Chart 

Ohio Proficiency California Cleveland 

Test Skill Achievement Test Competency Test 

Objective (Level , objective ) (Grade , objective ) 



Compute , whole , 




I O , 1 / 11/ 


decimal , fractions 


V 18 , 59—70 ) 


Compare, order, 
equivalence 






\ io i Dy— /u / 


solve, use proportions 


ma 7 A \ 

(18,74/ 


f ft 3 s * > 


Rounding 


(18,73; 


/ ft flft \ 
( S , VP / 


Percentages 


( 18 , 71 / 


/ft 9ft \ 








and metric 


(18,77) 


(8,18) 


Measurement, conversion 


(18,77) 


(8,18) 


Scales, interpolations 


(18,77) 


(8,18) 


Lines and figures 


(18,78) 


(8,13) 


Perimeters 


(18,78) 


(8,20) 


Area, Volume 


(18,78) 


(8,21) 


Tables, charts, maps 


( n/a ) 


(8,31) 


Probability 


(18,76) 


(8,32) 


Compute Averages 


(18,76) 


(8,32) 


Number sentences, formulas 


(18,74) 


(8,29) 


Evaluate algebraic 






expressions 


(18,73) 


( n/a ) 



Based upon the assumptions that the CAT was originally 
selected and the Cleveland Reading Test was subsequently 
developed (appended to the CAT) because of the preferred match to 
the reading/Engiish-language arts and mathematics curricula 
adopted by the district, one would feel confident that tested 
learning outcomes on the ninth grade proficiency test were quite 
thoroughly addressed in Cleveland's reading and mathematics 
curricula . 

Performance Comparisons 

In order to correlate the results of the ninth grade 
proficiency testing completed in the 1990-91 academic year with 
the achievement testing implemented in the district during the 
previous academic year, available proficiency test results (by 
student) were matched with the district's test results files in 
order to obtain each students CAT scores and CRT scores for 
reading and mathematics. The following file characteristics 
illustrate the sample utilized for this study. 

Ninth grade students in the Cleveland City School 
District (n - 6207) were tested in November, 1990 for the first 
administration of the Ohio Ninth Grade Proficiency Test. A 



7 



7 

second administration of the test (an alternate form) vas 
undertaken in Ma^ch, 1991 for those students who failed any part 
of the test in the fall. Records for the two administrations 
during the academic year were merged to create the basis of the 
data for this study. A total of 4,611 records (74% of the 
enrollment) containing pass-fail information for each subject 
tested were posted onto the proficiency test file. Next, the 
file was matched with the District's testing files to obtain the 
most recent norm referenced and criterion referenced test scores 
in reading and mathematics. Watched records were obtained in 66% 
of the cases (n « 3071) for norm referenced reading, 88% (n - 
4080 of the cases for norm referenced math, and in 86% (n - 3973) 
and in 88% (n - 4u85) of the cases for criterion referenced 
reading and math respectively. 

The analysis was conducted to obtain Spearman Rank order 
correlations for the norm referenced stanine scores and the 
criterion referenced mastery rates with the Ninth Grade 
Proficiency test pass-fail designations. The following table 
illustrates the correlation coefficients obtained from the 
analysis . 

TABLE 3 

Correlation Coefficients of Student Performance 
on Assessment Instruments 

Ohio Ohio CAT CAT Cleve Cleve 
Rdg Math Rdg Math Rdg Math 

Ohio 

Rdg 1.00 0.32 0.55 0.44 0.49 0.4S 

Ohio 

Math 1.00 0.47 0.54 0.45 0.59 

CAT 

Rdg 1.00 0.56 0.76 0.61 

CAT 

Math 1.00 0.55 0.83 

Cleve 

Rdg 1.00 0.59 

Cleve 

Math 1.00 
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The coefficients obtained through the analysis of 
variable pairs for each case on the file are illustrated in Table 
3 above. All relationships tested were positive and significant 
(p < .01), although varying degrees of relationship strength were 
evident. The coefficients obtained between the Ohio Reading test 
performance and those of the California Reading Test and the 
Cleveland Reading test were not strong (0.55 and 0.49 
respectively). Mathematics comparisons were similar, however, 
slightly stronger. The coefficients calculated for the Ohio Math 
and California Math and the Cleveland Math tests showed positive 
and significant relationships (0.54 and 0.59 respectively) like 
reading. 

The comparisons studied yield indices of relationship 
which were weaker than anticipated based upon the relatively 
thorough match of objectives cited above. 

Socioeconomic Analysis 

Finally, a selected sample (n « 31) of geographically 
contiguous districts within Cuyahoga County (including Cleveland) 
were analyzed by correlating their rate of Ohio Proficiency test 
passage (among ninth graders) with their poverty rank. The rates 
are based upon data compiled by the Council for Economic 
Opportunity in Greater Cleveland. 

The comparisons yielded a strong positive correlation 
(rho».7315) between overall performance among pupils and the 
district's poverty rank. That is, the lower the poverty rank (or 
the greater percentage of the population below the poverty 
level), the lower the passage rate. The results were significant 
(p < .01) for the sample utilized for the analysis. 

Discussion and Conclusions 

The two phase analysis reported above supports the 
notion that the Ohio Ninth Grade Proficiency Test does represent 
a valid measure of ninth grade competencies of students within 
the Cleveland City School District. The comparison of tested 
proficiency skills with the district's stated competency 
objectives in the areas of reading and mathematics yields a 
comprehensive match which would indicate a valid skill 
assessment. However, correlations between the district's 
measures (both norm referenced and criterion referenced) of 
student performance with Ohio Proficiency Test measures of 
reading and mathematics performance are not strong. While all 
comparisons rendered significant results, the coefficients 
ranging from 0.49 to 0.59 do not represent the strong 
relationship which had been expected and do not support the 
observations mentioned above. Stated simply, if curricular 
objectives are thoroughly matched on all testing instruments, why 
is the performance not comparable on the three measures of the 
'same material'? The issue of item (and total test) difficulty 
must be explored. 
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The comparison of district level performance on the Ohio 
Proficiency test as a function of each district's poverty rates 
indicates a strong positive relationship exists between the 
economic status of the district's pupils and their performance on 
the test. This factor also contributes to the question of 
disparate impact of the stated uses of test results. 

The evidence compiled in this study indicates that the 
issues identified in the introduction remain unresolved. The 
Ohio Proficiency test (and its resultant graded diploma system) 
may very well be a discriminatory practice within the state. The 
question remains, how can a test which appears to measure 
appropriate skills among the state's ninth grade students render 
such disparate results from other valid measures of the same 
skills? Also, the issue of vastly disproportionate results from 
the state's economically deprived student population continues to 
pose the issue of test adequacy for all of the state's pupils. 

While the results lead to the inconclusive discussion 
above, further analysis remains appropriate. Discriminant 
analysis is the next strategy useful to investigate the 
relationship between the measures reviewed in this study, rhe 
ability to use district obtained measures to predict the 
student's performance on the state proficiency test may very well 
serve the district's curricular and instructional objectives of 
increased pupil performance as indicated by better graduation 
rates. 
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